Skip to content

Add Jupyter DAP handler#1171

Merged
lionel- merged 18 commits intomainfrom
notebook/debugger
Apr 29, 2026
Merged

Add Jupyter DAP handler#1171
lionel- merged 18 commits intomainfrom
notebook/debugger

Conversation

@lionel-
Copy link
Copy Markdown
Contributor

@lionel- lionel- commented Apr 25, 2026

Branched from #1170
Backend side of posit-dev/positron#13205
Part of posit-dev/positron#12104
Closes #572

Implements notebook debugging via the Jupyter Debug Protocol. DAP messages flow through debug_request/debug_reply on the Control socket, with events forwarded as debug_event on IOPub.

Since notebook cells aren't files on disk, the Jupyter Debug Protocol uses dumpCell to write cell source code to deterministic temp files (path derived from a MurmurHash2 of the code). The frontend sends dumpCell before setBreakpoints, so breakpoints reference these temp files.

When execute requests contain a cellId, breakpoints and line directives are mapped to the temp files written by dumpCell. This allows users to execute and inject breakpoints at the same time, outside of a debugging session. Otherwise they'd need to start a debugging session with Debug Cell, then go back and reexecute cells containing breakpoints.

In addition:

  • Interrupts in notebook mode now quit the debugger with Q if active. This fits the notebook UX because, unlike in Console mode, the kernel remains busy for the duration of the debugging session, and the notebook UI shows an Interrupt button next to the cell. If we don't quit the debugger, the cell will remain in busy state after interrupting, which is confusing.

  • Debugging prompts created via browser() or debug() (as opposed to breakpoints) map to StdIn. Not the best UI in Positron (stdin prompts show via command palette) but much better than doing nothing. The alternative would be to initiate a full debug session from the backend, but this would require non-standard frontend changes.

Screen.Recording.2026-04-25.at.11.11.03.mov

@lionel- lionel- requested a review from DavisVaughan April 25, 2026 09:23
@lionel- lionel- force-pushed the notebook/debugger branch 9 times, most recently from f3d3154 to 5121d8c Compare April 26, 2026 07:58
Copy link
Copy Markdown
Contributor

@DavisVaughan DavisVaughan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not give dap_notebook.rs or dap_jupyter_handler.rs a too close read, but I did read everything else.

Mostly im not really sure i agree with the tradeoff of all of the added weird complexity that comes with debug-via-stdin in the browser() or debug() case, vs the straightforward support we have for breakpoints here. I think the UI for the former is not very good, and it might be better to just do nothing in that case and encourage people to use breakpoints in notebooks instead. To me the complexity increase is not worth it for the miniscule amount of people who might use that feature. (I have another comment on this below)

Comment thread crates/amalthea/src/fixtures/dummy_frontend.rs Outdated
Comment thread crates/ark/src/shell.rs
status: Status::Ok,
banner: kernel_info.banner.clone(),
debugger: false,
debugger: true,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, i guess we just send it anyways

    # A boolean flag which tells if the kernel supports debugging in the notebook.
    # Default is False.
    # Deprecated as replaced by 'supported_features'=['debugger'] (see below).
    'debugger': bool,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep we're sending both

Comment thread crates/ark/src/dap/dap_state.rs Outdated
Comment thread crates/ark/src/control.rs
if matches!(self.session_mode, SessionMode::Notebook) {
let dap = self.dap.lock().unwrap();
if dap.is_debugging || dap.is_debugging_stdin {
drop(dap);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the explicit drop() meaningful?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like good practice

Comment thread crates/ark/src/control.rs Outdated
Comment on lines +30 to +32
dap: Arc<Mutex<Dap>>,
session_mode: SessionMode,
dap_handler: DapJupyterHandler,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since all of these are only used for notebooks, I was kind of confused that they are here at all for console mode.

It might be nice to wrap these up in an optional struct, like

struct JupyterDap {
    dap: Arc<Mutex<Dap>>,
    dap_handler: DapJupyterHandler,
}

which is then Option<JupyterDap> on Control

and then you can use the precence of that field rather than matches!(self.session_mode, SessionMode::Notebook) to make decisions in other parts of the codebase

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm intending to get breakpoint sync via the jupyter handler even in Console mode.

For now I made the handler optional, but in the future we'll have a specialised handler for consoles too.

Comment on lines +634 to +644
events.push(DapBackendEvent::BreakpointState {
id: bp.id,
line: bp.line,
verified: true,
message: None,
});
}
}

for event in events {
self.send_backend_event(event);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be cleaner to just wrap DapBackendEvent::BreakpointState { in self.send_backend_event()?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You avoid the events vec alltogether, I think

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a comment explaining why it's structured this way:

        // Collect events first: `bp_list` borrows from `self.breakpoints`,
        // which prevents calling `&mut self` methods like `send_backend_event()`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting that feeling that iopub_seq should be wrapped in a Cell or something like that so it can be interiorly mutable allowing send_backend_event() to just be &self, because it really doesn't feel like you should need &mut self to do a send_backend_event(), and the counter feels like an internal detail

let Some(bp) = bp_list.iter_mut().find(|bp| bp.id.to_string() == id) else {
return;
};
let event = {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a very unnecessary extra { scope?

Copy link
Copy Markdown
Contributor Author

@lionel- lionel- Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

borrow-checker

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope

    pub fn verify_breakpoint(&mut self, uri: &UrlId, id: &str) {
        let Some((_, bp_list)) = self.breakpoints.get_mut(uri) else {
            return;
        };
        let Some(bp) = bp_list.iter_mut().find(|bp| bp.id.to_string() == id) else {
            return;
        };

        if !matches!(bp.state, BreakpointState::Unverified) {
            return;
        }

        bp.state = BreakpointState::Verified;

        let event = DapBackendEvent::BreakpointState {
            id: bp.id,
            line: bp.line,
            verified: true,
            message: None,
        };

        self.send_backend_event(event);
    }

})
})
.log_err();
.collect();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could probably avoid the collect() and just do .into_iter() on the filter_map result and loop over that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

borrow-checker

/// This must match the client-side `PathEncoder` in
/// `positron-runtime-debugger` so that the notebook location mapper can
/// correlate cell URIs with runtime source paths.
pub fn murmur2(data: &[u8], seed: u32) -> u32 {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could also just use someone else's?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also implement it on the frontend side. We could use the murmur2 crate, or just keep this one. I'm going with the latter since it's tested already.

Comment on lines +24 to +34
/// The temporary file prefix, cached at first access to avoid `TMPDIR`
/// instability. On macOS, R may unset `TMPDIR` during startup, causing
/// `std::env::temp_dir()` to return `/tmp` instead of the per-session
/// `/var/folders/.../T/` directory.
static TMP_FILE_PREFIX: LazyLock<String> = LazyLock::new(|| {
let mut tmp_dir = std::env::temp_dir();
let pid = std::process::id();
tmp_dir.push(format!("ark-debug-{pid}"));
// Trailing separator so the prefix can be concatenated directly with
// the hash and suffix (e.g. `{prefix}{hash}.r`).
format!("{}/", tmp_dir.display())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like the tempdir crate for this kind of thing?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we're creating a directory with a very specific pattern that remains valid for the whole session.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can do all that with the tempfile crate (not tempdir, sorry)

But regardless, it looks like its default is to just use std::env::temp_dir as the starting location to put the tempdir in, so I don't think it really matters

But based on your comments im now interested in the timing - do we need to trigger this lazylock before we start R up?

@lionel-
Copy link
Copy Markdown
Contributor Author

lionel- commented Apr 29, 2026

Mostly im not really sure i agree with the tradeoff of all of the added weird complexity that comes with debug-via-stdin in the browser() or debug() case, vs the straightforward support we have for breakpoints here.

I think it's worth it because it fixes a longstanding limitation of Ark in Jupyter apps, where browser prompts have to be nested within a given execute request (i.e. a cell). Also I do think it can be useful in Positron too, albeit for advanced cases.

I would also not overstate the added complexity of that stdin support, I'm comfortable with it especially with the test coverage.

Screen.Recording.2026-04-29.at.14.13.37.mov

@lionel- lionel- force-pushed the notebook/dap-abstraction branch from 37d682d to e9eef70 Compare April 29, 2026 13:22
Base automatically changed from notebook/dap-abstraction to main April 29, 2026 13:34
@lionel- lionel- force-pushed the notebook/debugger branch from 5121d8c to 5c619b8 Compare April 29, 2026 15:22
@lionel- lionel- force-pushed the notebook/debugger branch from 5c619b8 to 413eaf0 Compare April 29, 2026 17:51
@lionel- lionel- merged commit e53a623 into main Apr 29, 2026
12 of 13 checks passed
@lionel- lionel- deleted the notebook/debugger branch April 29, 2026 17:51
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Emit debug prompts on StdIn in notebook mode

2 participants